Computer Vision Applied to Super-resolution

Super-resolution restoration aims to solve the following problem: given a set of observed images, estimate an image at a higher-resolution than is present in any of the individual images. Where the application of this technique differs in Computer Vision from other fields, is in the variety and severity of the registration transformation between the images. In particular this transformation is generally unknown, and a significant component of solving the superresolution problem in Computer Vision is the estimation of the transformation. The transformation may have a simple parametric form, or it may be scene dependent and have to be estimated for every point. In either case the transformation is estimated directly and automatically from the images. Computer Vision techniques applied to the superresolution problem have already yielded several successful products, including Cognitech’s “Video Investigator” software [1] and Salient Stills’ “Video Focus” [2]. In the latter case, for example, a high resolution still of a face, suitable for printing in a newspaper article, can be constructed from low resolution video news feed. The approach discussed in this article is outlined in figure 1. The input images are first mutually aligned onto a common reference frame. This alignment involves not only a geometric component, but also a photometric component, modelling illumination, gain or colour balance variations among the images. After alignment a composite image mosaic may be rendered and super-resolution restoration may be applied to any chosen region of interest. We shall describe the two key components which are necessary for successful super-resolution restoration : the accurate alignment or registration of the low-resolution images; and the formulation of a super-resolution estimator which utilizes a generative image model together with a prior model of the super-resolved image itself. As with many other problems in computer vision, these different aspects are tackled in a robust, statistical framework.